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ABSTRACT 

In this paper we report some results of the analysis of 
released items of the Third International Mathematics and Science Study 
(TIMSS) . This analysis is part of a research project in process related to 
Mexican high school curricula from an international perspective and, the 
mathematics students* performance in this school level. The results obtained 
up to now are related to the mathematical curriculum content underlying the 
set of TIMSS assessment items. The findings suggest a great wealth of 
mathematical contents and some skills and abilities needful of answering to 
this. set of items. However, in this paper we report only the results 
corresponding to mathematical content, highlighting some detected differences 
between the Mexican high school curricula and these contents. (Author) 
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Abstract: In this paper we report some results of the analysis of released items of the 
Third International Mathematics and Science Study (TIMSS). This analysis is part of 
a research project in process related to Mexican high school curricula from an inter- 
national perspective and, the mathematics students’ performance in this school level. 
The results obtained up to now are related to the mathematical curriculum content 
underlying the set of TIMSS assessment items. The findings suggest a great wealth of 
mathematical contents and some skills and abilities needful of answering to this set of 
items. However, in this paper we report only the results corresponding to mathemati- 
cal content, highlighting some detected differences between the Mexican high school 
curricula and these contents. 

Introduction 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



The need to know how young people are being prepared at school leads us to the 
evaluation of student performance as well as instruction, curricula and teacher prac- 
tices. The Third International Mathematics and Science Study (TIMSS), conducted 
in 1992-1995, and its repetition (TIMSS-R), in 1999, was the most extensive and 
ambitious international mathematics and science evaluation of comparative education 
achievement ever undertaken (Michel, & Kelly, 1 996; Gonzales et al., 2000). The main 
goal was to examine student achievement in some school grade levels to try to under- 
stand the nature and extent of student achievement and the context in which it occurs 
(Michel & Kelly, 1996). 

For TIMSS implementation, analysis and data collection, different instmments 
were prepared. They included questionnaires and interviews for students, teachers and 
principals of their schools, videotapes, classroom activities and case studies of three 
countries, curriculum analyses of participant countries, and assessment item sets for 
students (Mullis & Michel, 1996). 

One of the most important tasks carried out by TIMSS was the design and selection 
of the item sets in such a way that this would reflect the knowledge and some skills 
and abilities that students should acquire and develop according to their age and 
school grade level. For this purpose, the TIMSS team took into account the analysis 
of curriculum guides, textbooks and other material that allowed them to determine 
the common characteristics of most of the participant countries. 
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In this paper we report some results of a research project in process related to 
Mexican high school mathematics curricula from the international perspective, implic- 
itly suggested by the TIMSS assessment instruments. This research project began with 
the analysis of the TIMSS instruments from different perspectives: 

Determining the underlying content in the set of items to identify a curriculum, 
which we can refer to as international, at the content level. This international 
curriculum can serve as a reference for the curriculum reform in Mexican 
high school mathematics education, since it will allow us to compare the 
Mexican mathematics curriculum with this broad opinion of specialists in both 
mathematics and mathematics education world-wide. 

Identifying particular skills and abilities, procedures and solution strategies for 
each item. Doing this we can have a broad picture of the things that can be 
assessed with this set of items. This information allows us to have a richer 
curriculum that includes not only content knowledge but also processes, abilities 
and skills. 

• Designing instruments complementary to those of TIMSS to identify more 
clearly the knowledge, skills and abilities that students should develop and 
acquire. The enrichment of these items by means of adaptation and incorporation 
of new ones, will allow us to investigate more deeply into the knowledge, 
skills, and abilities identified in the original instruments, seeking to have a more 
profound diagnostic of Mexican high school mathematics education. 

Under the assumption that the TIMSS set of items reflects a desirable curriculum 
for high school systems from an international perspective, we intended to determine 
the common contents and differences between the Mexican high school and this inter- 
national curriculum. We suppose that Mexican high school curriculum contents cannot 
be very different from the one outlined by this international perspective, since it is not 
isolated from international expectations, mainly due to the current global tendency in 
which we are immersed. 

Assessment of Knowledge, Skills and Abilities 

One fundamental stage in the design and analysis of items to assess student per- 
formance is to know accurately what it is to be assessed. To be clear in this aspect, it 
is desirable to distinguish between knowledge, skills and abilities. 

Knowledge acquisition is a process that involves both the memorization of facts, 
principles and laws and the appropriation of mathematical concepts (Haladyna, 1 997). 
Although the purposes and goals of mathematics education in most of the Mexican 
educational systems are guided to promote the development of competencies and 
skills (SEP, 1999), the assessment has been focused mainly on content knowledge. 

In many countries, however, there is a growing interest in the way that we are assess- 
ing student performance. For example. The Principles and Standards for School Math- 
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I ematics suggests that assessment “should focus on students’ understanding as well as 
I their procedural skills” (NCTM, 2000, p. 23). 

i , . Skills and abilities are cognitive processes that can be inferred by means of study- 
ing the students’ behavior when they are carrying out a certain activity. Skills are spe- 
;cific actions that are characteristic of the activity, while abilities are the psychologi- 
cal characteristics of that person who carries out the activity (Krutetskii, 1976). These 
. aspects have been attended to little in the assessment process, not because they are 
:seen as less important, but because of the difficulty in studying them. It is important to 
mention, however, that currently this is a central part of some mathematics education 
research. It seems to be that a qualitative inquiry methodology applied to the assess- 
ment process is an appropriate way to investigate these issues (Romberg & Collins, 
2000 ). 

lOiowledge, skills and abilities are not acquired and developed in an isolated way, 
they interact and develop in a combined form (Strenberg, 1999). 

Taking into account these aspects, our analysis is focused on the identification of 
the mathematical contents reflected by the TIMSS items, which in turn give us a well- 
rounded picture of the knowledge involved. 

Methods and Procedures 

From the TIMSS perspective, curriculum was considered to have three manifesta- 
tions: 

• The intended curriculum, that is, the mathematics that one expects students to 
study and learn in school; 

• The implemented curriculum refers to what the teacher actually teaches, in terms 
of their own interpretation of the intended curriculum; and, 

• The attained curriculum, which refers to what the students are really learning 
(Michel & Kelly, 1996). 

These manifestations tell us that official intentions can be different from what the 
students are really learning. Determining the underlying contents in the TIMSS items 
leads us to identify one part of a deliberate international curriculum that may not cor- 
respond to what the students are really achieving; this international curriculum, how- 
ever, integrates the expectations for students at an international level. 

The attained curriculum can be determined by assessing student achievement in 
those expectations marked by the intended curriculum. As we have mentioned, one 
of the purposes in determining the underlying* curriculum in the TIMSS item set is to 
have a broad picture of the knowledge, skills and abilities that we can assess with this 
pool of items. Their implementation will allow us to have a diagnostic of the actual 
situation of the Mexican high school system, and a framework with which to compare 
expectations and students’ performances in those contents that are common in both the 
Mexican and the international curricula. 
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In 1996-98 in Mexico a team of mathematics education researchers and math- 
ematics researchers, conducted the study ^Tropositos y Contenidos de la Ensenanza 
de las Matemdticas en el Nivel Medio Superior en Mexico'' (Rivera et al., 2000), which 
was focused on the intended curriculum across the country. This study shows the com- 
plexity and diversity of Mexican high school systems; it also highlights some schools 
systems that are representative of them. 

For our analysis, we used the results of this study and the 57 released TIMSS 
mathematics items that correspond to the high school system in Mexico. The data 
gathered by this means is appropriate for a qualitative approach; so we used this 
method of inquiry. 

Results 

TIMSS classified the items in various ways. One of them is by the format, mul- 
tiple choice and open-ended (short and extended response) items. 

The use of multiple-choice items has some advantages, for example, the ease 
in assigning and applying scores, mainly when we want to test a large population. 
The main purpose to using them in assessing student performance is to determine the 
knowledge acquired as a product of the learn process (Haladyna, 1997). This kind 
of format, however, is not appropriate for assessing some skills and abilities, since 
these are aspects that can only be evaluated by analyzing the solution process of a 
problem or the students’ behavior when facing an activity. Others that, besides measur- 
ing the students’ knowledge base, also try to develop certain competencies are begin- 
ning to substitute multiple choice items, which is in widespread use in standardized 
tests. Examples of this are the performance assessment packages that consist of com- 
bined-solving problems and/or open-ended problems. 

The item format gives us insight into the extent of the answer that students should 
give us, and this extent reflects the depth of the assessment process, so this is an aspect 
that we take into account in the analysis. 

TIMSS classified the items in the Final Year of Secondary School test (final year 
of high school in Mexico) into mathematics literacy and advanced mathematics item 
sets. The former was to be applied to a sample of all students in the last year of second- 
ary school, while the advanced mathematics test was to be applied to students who 
were taking advanced mathematics courses in their final year of secondary school. The I 
items in the latter were classified in turn as numbers, equations and functions, geom- ; 
etry, calculus, probability and statistics, and validation and structure. | 

Although the mathematics literacy items were not explicitly classified in more t 
detail, our analysis suggest the following more relevant contents: \ 

• Numbers and number sense: Whole numbers and rational numbers, rational 

numbers representations in decimal and fractional forms and their relationships, ' 
basic operations with whole numbers and rational numbers, basic operation j 
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properties, unit conversion, reasonableness of results, order of magnitude, 
estimation, rounding, number patterns, and various types of percent problems. 

• Proportionality: Direct and inverse proportionality, ratio, and scales in graphs. 

• Probability and statistics: Data representation; interpreting graphs, plots and 
charts, prediction and inference from graphs and charts, notions of sample 

and population, randomness, measures of central tendency (mean), fitting lines, 
numerical probability; notions of mathematical expectation, use of notation and 
vocabulary. 

• Variables and functions: Dependence between variables, function graphs, inter- 
preting graphs of functions, relationship between graphs and equations, recall 
and use of formulas, mean rate of change, and increments. 

• Geometry: Longitude, area and volume measurements, graphic estimation, and 
recall of names of basic figures (triangle, square, rectangle, hexagon, cube). 

For the advanced mathematics items, we identified the content knowledge in each 
one of the TIMSS categories: 

• Numbers, equations and functions: Real and complex numbers, basic 
operations with these numbers, conjugation of complex numbers, operational 
properties, exponents and radicals (square root), prime factors, permutations 
and combinations, direct and inverse relationships between variables, variation 
range, approximation, estimation, rounding, precision, inequalities, posing and 
solving equations with real and complex numbers, linear systems of equations, 
rearranging formulas, equations of conic curves and equations of planes, 
logarithmic exponential and trigonometric functions, use of specialized notation 
and vocabulary. 

• Calculus: Routine computation of limits with radicals (square root), the 
derivative and the integral (in speed and acceleration context, and their 
relationship), the derivative as a function (graphic representation and their 
relationship with the function), graphic representation of the integral as a 
primitive function, graphic representation of the integral as the area of a region, 
exponential and logarithmic functions and their relationship, trigonometric 
function and the use of some trigonometric formulas in problem solving 
situations, series (geometric and arithmetic), and the use of a more specialized 
vocabulary and notation. (We also distinguished some algebraic processes, like 
the conjugated product of bihomials, and basic algebraic operations). 

• Geometry: Representation of points on a plane, the distance between two points 
on the plane, longitude, ^ea and volume measurement and units, properties 

of triangles (exterior and interior angles, the sum of interior angles, inscribed 
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nght triangles in the circumference, congruency and similarity), parallelism and 
perpendicularity, regular polygons (their area), conic sections (identification of 
equations and graphics), Pythagorean theorem applications, lines and planes in 
space, graphic estimation of an area, spatial visualization, coordinate systems 
in two and tree dimensions, translation, reflection, rotation, invariance, vectors, 
vector representations on a plane, angle between vectors, sum, difference and ' 

product by scale operations with vectors, and correct use of a more specialized 
notation. ^ 

Probability and statistics: Data representation and analysis, interpreting graphs, 
plots and charts, process of randomness, measures of central tendency, counting 
principles, scales, estimation, prediction and inference, classic probability, 
conditional probability, numerical probability; fitting lines, correlation, and 
specialized vocabulary. 

Validation and structure: There is only one item in this category related to formal 
logic. 

The rnathematics literacy items and the advanced mathematics items differ con- 
siderably in quantity, context, notation and vocabulary and content knowledge. For 
instance, the items related to geometry in the literacy mathematics test are fewer, and 
niost of them are related to the names of geometric figures and measurement. In the 
advanced mathematics test, most of them are related to the properties of geometric 
figures, representations and spatial visualization. 

Most of the Items in the literacy test are in context; they incorporate situations and 
activities from daily life, while most of the items in the advanced mathematics test are 
in a mathematical context, or in a context that may not be very familiar to students; for 
example, item L3 related to the decomposition of a radioactive element. 

The Mexican high school system is complex. There is a diversity of subsystems 
both public and private. There are no standards or benchmarks to support the curricu- 
lum design and therefore, we found a diversity of curriculum guides. Although all of 

them include mathematics, some only in one part of the program and others during the 
whole program. 

Within this diversity, however, there exist common mathematical contents, related 
to algebra, tngonometry and analytical geometry (Rivera et al., 2000). These contents 
relate well to the international curriculum that we described above (although TIMSS 
does not refer explicitly to algebra contents, some of them correspond to the algebra 
courses in Mexico); for example, solution of linear equations, trigonometric functions 
in problems that involve right triangles or the study of conic sections. 

In general terms, our findings show that large differences do not exist between the 
curriculum contents in Mexican high schools and the international curriculum. There 
are, however, some differences. For example, numeric and graphic estimation is pres- 
ent in many of the TIMSS items, both in the advanced mathematics test and in the 
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literacy test, but this notion was not identified in most of the Mexican high school cur- 
riculum contents. 

Also, we found differences in the treatment of certain concepts. For example, 
item K5 in the advanced mathematics test refers to the identification of relationships 
between the function and their first and second derivative in a graphic representation 
(and some data in analytical representation). There are no instances of this type of 
treatment in the high school curriculum in Mexico. The derivative concept is present 
in the calculus courses (In Mexico, the students that take calculus courses, are those 
who expect to enter to college in areas related to mathematics and science studies or 
engineering). The treatment of this concept is in terms of the application of formulas 
to solve routine problems. The graphic representation of the derivative is only used as 
a way of introducing this concept in an intuitive form, such as for the slope of a tangent 
straight line to the graph of a function in a point. The second derivative is not present 
in the Mexican high school curriculum. 

Although the content is only one part of the curriculum, it is an important one 
that allows us to appreciate the wealth of content knowledge and the expectations for 
students. The framework that TIMSS provides us is an excellent opportunity to reflect 
on these expectations. 
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